Patch Based Confidence Prediction for Dense Disparity Map

Figure 1: Overview of our method. Confidence of stereo correspondences is useful information to improve quality of the disparity maps. Many features to predict the confidence have been proposed. Learning based confidence measure[6] combines these features and is able to outperform their individual usage. These features and classifiers are carefully designed, however beneficial information might be undescribed or their representation might be too redundant. As shown in Fig.1, we propose a novel confidence prediction method to overcome the problem. We design a disparity patch which takes into account the ideas of conventional confidence features. The patches are used as inputs of a Convolutional Neural Network(CNN) so that the discriminative features and classifier are simultaneously trained. In order to handle trade-off between accuracy and computation time, we propose three types of network structures and their input patches. Moreover, the confidence is incorporated into Semi Global Matching (SGM) [4] to acquire dense disparity map. SGM is widely used for dense disparity estimation due to its high accuracy while keeping low computation cost. In the following, we will briefly explain both methods and experimental results. Confidence estimation with a CNN: We leverage the disparity patch and introduce the knowledge of the conventional features. The patch consists in a two channels. 1st channel is coming from an idea that neighboring pixels on a disparity map D1 which have consistent disparities are more likely to be correct matching[7]. In 2nd channel, a disparity D2 from another image is considered such that the matches from left to right image should be consistent with those from right to left[1]. We employ a shallow CNN for the sake of reducing potential computation cost of the network, however, the network is still slow computation because the output of the network for each pixel has to be computed from scratch. We also propose speed-up networks by modifying preprocessing of the patches and network structure. Confidence fusion with SGM: SGM has two parameters in order to control discontinuities of disparity map. We assume the discontinuities are likely to have the large magnitude of the image gradient using the same assumption as the original SGM, but not all large gradient pixels correspond to them. We consider the pixels with high confidence should be trusted and are able to be discontinuities easily. Hence, penalties at the high confidence pixel are designed to be decreased. Figure 2 and Table 1 show evaluation results based on sparsification curve and its area under curve (AUC) value. Better confidence prediction methods have AUC values that are closer to the optimal curve: It means the method removes incorrect correspondence pixels while keeping the Sparsification